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= What is research reproducibility? 


= Concerns regarding research reproducibility in biomedical 
research 


= What is currently being done to address issues of 
research reproducibility? 


= Can librarians help to address research reproducibility 
concerns in biomedical research and if so how? 


= What impact could librarians have? 


= Key messages 
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= Terminology not always consistently or correctly used leading to 
lack of clarity and confusion 


= The National Science Foundation (NSF) defines 
‘reproducibility’ as 


‘the ability of a researcher to duplicate the results of a prior study 
using the same materials and procedures as were used by the 
original investigator". 


= The NSF states that 


"Reproducibility is a minimum necessary condition for a finding to 
be believable and informative’. 


Attps://www.nst.gov/sbe/SBE_ Spring 2015 AC Meeting Presentations/Bollen_Report_on Replicability_ SubcommitteeMay 2015. pdf 
Accessed 3 May 2017. 
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= feplication’ & ‘replicability’ are terms often used when 
reproducibility is discussed 


= but there is a distinct difference in meaning between 
‘reproducibility’ and ‘replicability’ 


= "Replicability' is defined by the National Science 
Foundation as 


"the ability of a researcher to duplicate the results of a prior 
study if the same procedures are followed but new data are 
collected”. 
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Science 101: the basics of reproducibility 


Brian Nosek, Psychology Professor at the University of Virginia and the President 
and Director of the Center for Open Science 





nA 
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https://www.youtube.com/watch ?v=lvw4HBI8Lyo 
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= 4 main aspects of reproducibility: 


— Methods (detailed enough description of each step of the 
study methods and data for it to be repeated) 


— Results (can the results of the study be replicated) 


— Inferential (can groups performing analysis on the same data 
reach the same conclusions) 


(first three based on Goodman SN, Fanelli D, Ioannidis JP. What does research reproducibility mean? Sci Transl 
Med. 2016 Jun 1;8(341):341ps12). 


— Laboratory methods and materials (detailed enough 
information about the materials (e.g. cell lines) and laboratory 
methods (e.g. conditions, storage) to allow them to be 
accurately reproduced) 





Concerns about reproducibility in 





biomedical research (1) @ equator 


Estimating the reproducibility of psychological 
science 


“A large portion of replications 
produced weaker evidence for 
the original findings despite 

using materials provided by the 


Open Science Collaboration + 
+ See all authors and affiliations 


Science 28 Aug 2015 
Vol. 349, Issue 6251, aac4716 
DOI: 10.1126/science.aac4716 


original authors, review in 
advance for methodological 
fidelity, and high statistical power 
to detect the original effect sizes” 
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Editorial 


Biomarkers 
in Medicine 


For reprint orders, please contact: reprints@futuremedicine.com 


Reproducibility in biomarker research and 


clinical development: a global challenge vy 
“Implementation of an academic quality system will contribute to 
a better understanding of and trust in the quality and validity of 
research results...” 
-PLOS | BIOLOGY Browse Publish Ab: 


Ú OPEN ACCESS fÈ PEER-REVIEWED 


META-RESEARCH ARTICLE 






Reproducible Research Practices and Transparency 


"When asked about questionable research practices, survey respondents were aware of across the Biomedical Literature 


other researchers who selectively reported study outcomes (41%) and experimental 


Shareen A. Iqbal $, Joshua D. Wallach E. Muin J. Khoury, Sheri D. Schully, John P. A. loannidis [& 


conditions (36%), adjusted statistical analysis to optimise results (43%), and engaged PV»!shed: January 4, 2016 + https://doi org/10.1371/journal pbio 1002333 


in other shady practices (20%). Fewer respondents admitted to engaging in these 
practices themselves, although 25% admitted to adjusting statistical analysis to 
optimize results. There was strong agreement that such practices should be reported 


in research papers..." 


Questionable science and reproducibility in electrical 
brain stimulation research 


Martin E. Héroux, Colleen K. Loo, Janet L. Taylor, Simon C. Gandevia fæ 


Published: April 26, 2017 ə https://doi.org/10.1371/journal.pone.0175635 





nature.com » journal home » archive » issue » perspectives » opinion » full text 





NATURE REVIEWS CLINICAL ONCOLOGY | PERSPECTIVES | OPINION 






Reproducibility of research and preclinical 
validation: problems and solutions 


Lajos Pusztai, Christos Hatzis & Fabrice Andre 


Affiliations | Contributions | Corresponding author 


Nature Reviews Clinical Oncology 10, 720-724 (2013) | doi:10.1038/nrclinonc.2013.171 
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LAE LANGEL | cswshoatanedialacene ches 


Biomedical research: increasing value, reducing waste 


Hatonim F balei | Brean Mr hip, Lan Egberts, Lirih [Hrg Lake hama, ohn PA board, Fy 
Suara, Aun ien Ohia, Paul ilini 


How to increase value and reduce waste when research 
priorities are set 


isin Chakners, Dee —, Prod Michel Ei Gracken, PhD, Prol Ben Dyulbegoetc, Pal, Sako Gargttind MD, borg) 
PAD, A Metin Gobreenagiu, Ph, Dawid W Hirama, Phil, Pet Johan PA kupari, WE, Sareh Han, Phi 


E rii 


increasing value and reducing waste in research design, 
conduct, and analysis 


Prof John iP A monn, weet * Prot Se Creed, WPH, Prol kark A ee BD, bhain J Khary, Dl 
Maloole R Medeni, PRD, Prol id Moher Phi, Prol itenneth F Shuir, PhD, Prod Sobe Tibahirani, PD 





ES 


Increasing value and reducing waste in biomedical research 
regulation and management 


Poo! Festa Ad Shahi Salas, PaDEe — Elaine Selle, MAppSiet Josathan Magen, PHD, Prod Elin Hereninki MI, 
Robert 5 Phillips, GM ICh, Prol Jalan tavulewce, PhD, Prol Misicokn buini, PhO, Lace Winely, PhO, lake Chulimer 


E irie 


increasing value and reducing waste: addressing inaccessible 
research 


De A Wen Than, Deh l Peo! Fujian Song, PRO, Andie Micken, PhD, tom Jeferson, MO, Pes! Key Deckerdin, Ph 
Pral Peter C Galicie, Dekel, Peel! parla H rel, BD, Cure hara, Phi, H Barl vat der Wei, Ph 
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Reducing waste from incomplete or unusable reports of 
biomedical research 


Poof Poel Glawrion, FRADGPER Prod Doggles G -Aiman, Der. Prod Patek Bossy, Pr, Prod babe Bourton, A 
Poof Mi Clarke, Pied, Prol Shean Julie, PhD, Prod saran Hicbir, OP hil, David Moher, PhD, Uilrabeth Wager, Pho 
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Many factors contribute to irreproducible research studies. 


= Key study-related factors include: 
— poor study design 
— inadequate sample size 
— use of inappropriate statistical analysis techniques 
— poorly controlled experimental conditions 


— lack of detailed documentation/recording of study 
procedures/methods (e.g. laboratory notebook) 


— poor reporting of study design and methods in publications 
— selective reporting of results 
— unavailability of data 


= Key external factors include: 
— pressure to publish 
— peer review 
— lack of outlets for publishing ‘negative’ results 
— conflicts of interest 
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nature International weekly journal of science 





= Nature conducted an online survey on reproducibility in research. 1,576 
researchers took part. Results published in May 2016. 


"= 52% of survey respondents answered the question ' is there a 
reproducibility crisis?’ with "Yes, a significant crisis’. 


= 34% of respondents answered ‘no’ to the question ‘have you established 
procedures for reproducibility?’ 


Baker M. 1,500 scientists lift the lid on reproducibility. Nature. 2016;533:452-454. Accessed on 4 May 2017. 
http://www.nature.com/news/1-500-scientists-lift-the-lid-on-reproducibility-1.19970 








Concerns about reproducibility in ESS 
biomedical research (4) @ equator 


network 








WHAT FACTORS CONTRIBUTE TO 
IRREPRODUCIBLE RESEARCH? 


Many top-rated factors relate to intense competition 
and time pressure. 


@ Always/often contribute ® Sometimes contribute 


Selecti rti “Loy: : 
ouve reporting Nature survey on reproducibility in 


Pressure to publish 





research. 1,576 researchers took part. 


Low statistical power 
or poor analysis 


Not replicated enough 
in original lab 


Insufficient 
oversight/mentoring 


Methods, code unavailable 


Poor experimental design 


Raw data not available 
from original lab 


Fraud 


Insufficient peer review 
Problems with 
reproduction efforts 


Technical expertise required 
for reproduction 

Variability of 

standard reagents | 


Bad luck 


Baker M. 1,500 scientists lift the lid on reproducibility. Nature. 
2016;533:452—454. Accessed on 4 May 2017. 
://www.nature.com/news/1-500-scientists-lift-the-lid-on- 





reproducibility-1.19970 





nature 
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= Key area of concern: research reporting 


— formal publications 
pre-print services 

— institutional repositories 
data sharing services 


= Examples directly relating to study reproducibility include 


— incomplete reporting 
unclear reporting 

— selective reporting 
misleading reporting 
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= Hundreds of published 
reviews show that key 
elements of method's and 
findings are commonly 


ne Ä 
missing from journal reports Ieeh 2A 
s THEN A K eN 


ap MIRACLE a m 






OCC URS 


“| think you should be more explicit here in 
step two.” 


from What’s so Funny About Science? by Sidney Harris (1977 
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What is missing from descriptions of Exercise prescription: a case for standardised 
A- : reportin 
treatment in trials and reviews? stati 
Repl cating non-pharmacolog: cal treatments in practice depends on how well they Clin Chem Lab Med 2012;50/3):411 mo 2012 by Walter de Gruyter + Berlin » Boston DO! 10. 1616/eelm 2011 0904 a 











have been described in research studies, say Paul Glasziou and colleagues 


Have you ever read a trial of revie 
woodered exactly how to car 
as è “behaviceral & 


i AG cs mgen lest eww cel An appeal to medical journal editors: the need for 
= ae a full description of laboratory methods and specimen 
handling in clinical study reports 





salt redacooa,” = “exerche progr ? H 7 
Akbough CONSORT and related ini- lashed a sepplemensary article with a 


{PLOS | ov 
Adequacy of Published Oncology Randomized 
RESEARCH ARTICLE Controlled Trials to Provide Therapeutic Details 


The Devil Is in the Details: Incomplete Needed for Clinical Application 
Reporting in Preclinical Animal Research | 


Marc T. Avey"2*, David Moher, Katrina J. Sullivan’, Dean Fergusson’, Gilly Griffin’) Jennifer M. Duff, Helen Leather, Edmund O. Walden, Kourtney D. LaPlant and Thomas J. George Jr 
Jeremy M. Grimshaw", Brian Hutton '®, Manoj M. Lalu’, Malcolm Macleod”, 
F. Turgeon®™", Lauralyn Mcintyre™", Canadian Critical Care Translational Biology G 


Reproducibility of Search Strategies Is Poor in Systematic | Non-compliance with randomised allocation and missing outcome 
Reviews Published in High-Impact Pediatrics, Cardiology data in randomised controlled trials evaluating surgical interventions: 
and Surgery Journals: A Cross-Sectional Study a systematic review 





Jonathan B. Koffel Eg f, Melissa L. Rethlefsen Temitope E. Adewuyi XZ , Graeme MacLennan and Jonathan A. Cook 


Published: September 26, 2016  http://dx.doi.org/10.1371/journal.pone.0163309 BMC Research Notes 2015 8:403 | DOI: 10.1186/s13104-015-1364-9 | © Adewuyi et al. 2015 





Example: intervention description aE 





Cluster RCT of 3 educational interventions to improve detection e ga eh 
and management of dementia in primary care (BMJ 2006;332) 

















Educational interventions 





We tested three educational interventions: an electronic tutorial on CD Rom, decision support software, and 
practice based workshops with a standard curriculum designed by a multidisciplinary expert group. The educational 
interventions reflected different approaches to adult learning: the electronic tutorial for self directed learning; 

decision support software for real time, real case learning; and workshops for peer reflection about real cases. Do you 


think this 
description 
is sufficient 





The electronic tutorial allowed for a mode of learning from case analysis, with the emphasis on reflecting on 
knowledge and revisiting particularly difficult and complex clinical problems. The tutorial is an “electronic book,” 
with an indexing system that allows easy access to different themes, and hypertext links that allow the reader to 
move easily from one subject to another. for the 


study to be 
replicated? 





The decision support software was written inside the existing electronic medical record software and produces 
prompts for the investigation and management of dementia. This mode of learning was driven by real cases, using 
computer decision support software to prompt and assist clinical reasoning and care planning in real clinical time. 


Two experienced general practitioners with backgrounds in postgraduate education facilitated the small group 









workshops with general practitioners and practice nurses. Case scenarios were discussed to illustrate appropriate 
management. This mode of learning involved case discussion in small, multidisciplinary groups. 





Control practices were visited only to collect data. Further details about the development, format, and piloting of 
the interventions are reported elsewhere." 15 


Example: intervention description aE 





Cluster RCT of 3 educational interventions to improve detection e gas rh 
and management of dementia in primary care (BMJ 2006;332) 


















Educational interventions 





We tested three educational interventions: an electronic tutorial on CD Rom, decision support software, and 
practice based workshops with a standard curriculum designed by a multidisciplinary expert group. The educational 


interventions reflected different approaches to adult learning: the electronic tutorial for self directed learning; Results: 


decision 


The electronic tutorial allowed for a mode of learning from case analysis, with the emphasis on reflecting on support system 


decision support software for real time, real case learning; and workshops for peer reflection about real cases. 





knowledge and revisiting particularly difficult and complex clinical problems. The tutorial is an “electronic book,” f and worksh op 
with an indexing system that allows easy access to different themes, and hypertext links that allow the reader to im prove 


move easily from one subject to another. detection rates 





The decision support software was written inside the existing electronic medical record software and produces 
prompts for the investigation and management of dementia. This mode of learning was driven by real cases, using 
computer decision support software to prompt and assist clinical reasoning and care planning in real clinical time. 


Two experienced general practitioners with backgrounds in postgraduate education facilitated the small group 









workshops with general practitioners and practice nurses. Case scenarios were discussed to illustrate appropriate 
management. This mode of learning involved case discussion in small, multidisciplinary groups. 





Control practices were visited only to collect data. Further details about the development, format, and piloting of 
the interventions are reported elsewhere." 15 
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Design and setting 

A prospective, randomised, double-blind controlled trial was 
carried out by one of the authors (KP) at Randfontein Estates 
Gold Mining and Westonaria Gold Mining from September 
1995 to July 1996. Patients were mineworkers. 


= A double-blind randomised trial means that the 
following groups were not aware of assignment to the 
intervention or control group: 


A. 


m J 


Participants and outcome assessors 
Participants and intervention providers 
Intervention providers and outcome assessors 
Any of the above 

None of the above 


-E H 
Example: study design description REESS 
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Design and setting 


A prospective, randomised, double-blind controlled trial was 
carried out by one of the authors (KP) at Randfontein Estates 
Gold Mining and Westonaria Gold Mining from September 
1995 to July 1996. Patients were mineworkers. 


= A double-blind randomised trial means that the 
following groups were not aware of assignment to the 
intervention or control group: 


A. Participants and outcome assessors 
B. Participants and intervention providers 
C. Intervention providers and outcome assessors 


D. Any of the above 


E. None of the above 


$3 
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Search strategy 
The design, analysis, and reporting for this meta-analysis followed the MOOSE guidelines [10]. Relevant studies were identified 


by a literature search of the PubMed database (from January 1966 to September 1, 2016), without restrictions, using the 
search terms “alcohol consumption”, “alcohol drinking”, or “alcohol intake” combined with “stroke”, or “cerebrovascular 
disease”, or “cerebral infarction”, or “intracerebral hemorrhage” or “subarachnoid hemorrhage”. The database search was 
performed by two authors (SCL and AWa) and enhanced by searches of the reference lists of identified articles. 


Is this description of the search strategy sufficient 
enough for the search to be reproduced confidently? 


Note 
(not looking at the quality or comprehensiveness of 


the search strategy just at whether or not the search 
as described could be confidently reproduced) 





C a U eee SSS 
Example: literature search description RESE 
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Search strategy 
The design, analysis, and reporting for this meta-analysis followed the MOOSE guidelines [10]. Relevant studies were identified 


by a literature search of the PubMed database (from January 1966 to September 1, 2016), without restrictions, using the 
search terms “alcohol consumption”, “alcohol drinking”, or “alcohol intake” combined with “stroke”, or “cerebrovascular 
disease”, or “cerebral infarction”, or “intracerebral hemorrhage” or “subarachnoid hemorrhage”. The database search was 


performed by two authors (SCL and AWa) and enhanced by searches of the reference lists of identified articles. 


No field tags given 


No indication of whether Mesh headings were used 
No full search strategy provided in appendix 
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= Key area of concern: research study methodology 


= Examples directly relating to study reproducibility include: 
— study design 
— sample size calculations 
Statistical analysis techniques 
experimental conditions 
documentation/recording of study procedures/methods 
data availability 


a) JOURNAL OF APPLIOD BEHAVIOR ANALYSIS 1974, 7,497 


THE UNSUCCESSFUL SELF-TREATMENT OF 
A CASE OF “WRITER'S BLOCK 


Dennis Urren 


VITERANS ADMINISTRATION NOSPITAL, BROCKTON, MASSACHUSETTS 


"Porton of this paper were non presented at the 
Bist Asmal Americus Prychologicgl Asociatia 
Canvention, Montreal, Canada, Augan 30, 1974. Re: 
prims may be obtmined irean Denai Upper, Behar 
Therapy Unis, Verran: Administration Hospital, 
Brockeon, Masschosems 02401. 


ccmiped JF Qeipber [97 5. 
(Padi imbar rin) 


COMMENTS BY REVIEWER A safficient detail to allow oder investigator to repli- 
I have studied his manuscripe very carefully with care De. Uppers badors. In compariom with the 
jema fide and tay itd hare not dieected a tingle dibar manascripts | ger from you comalnlng all char 
Haw in either design or writing syle. I suggest ic be © complicaned demil, chs one was a pleasure p examine. 
Poblished without revision Clearly it b che mom Surely we can find a place for this paper im che 
concise manuscript | have ever seex—yer it conning §Jownal—perhaps on che edge of a blisk pige 
497 
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COMMENTS BY REVIEWER A 
“I have studied this manuscript 
very carefully with lemon juice 
and X-rays and have not 
detected a single flaw in either 
design or writing style. | suggest 
it be published without 
revision. Clearly it is the most 
concise manuscript | have ever 
seen - yet it contains 

sufficient detail to allow other 
investigators to replicate Dr. 
Upper's failure. In comparison 
with the other manuscripts | get 
from you containing all that 
complicated detail, this one was 
a pleasure to examine. Surely 
we can find a place for this 
paper in the Journal-perhaps on 
the edge of a blank page”. 


SSS SSS 
Potential impact/consequences 5 
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= Impossible for other researchers to: 
— replicate methods 
— replicate the intervention 
— reproduce findings 


— or for readers even just to understand what was done and 
what was found by the research study 


= Research results cannot be translated into practice or used to 
inform future research 


= Waste of the time and money invested in the research study and 
can be considered unethical, particularly when patients have 
volunteered to take part 


= Consequences therefore are wide ranging and serious and can 
ultimately affect patient care 
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Current initiatives addressing research Wá er 
reporting and reproducibility (1) Sess 


= International campaigns and initiatives 


REWARD Alliance (http://researchwaste.net/) 
Lancet campaign (http://www.thelancet.com/campaigns/efficiency) 


EQUATOR Network (http://www.equator-network.org/) 
The Reproducibility Initiative 
(http://validation.scienceexchange.com/#/) 


The Reproducibility ABS Cancer Biology 
(https: /osf.io/e81x/wi i/home/) 

AllTrials (http://www.alltrials.net/) 

Center for Open Science (https://cos.io/) 


Global Biological Standards Institute Reproducibility 2020 action plan 
(https://www.gbsi.org/) 


METRICS Institute (https://metrics.stanford.edu/) 
NC3Rs Experimental Design Assistant (https://eda.nc3rs.org.uk/) 


Current initiatives addressing research 
reporting and reproducibility (2) 





National initiatives 
NIH HER and reproducibility (US) 





Sar UNIVERSITY OF 
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— Redan of Medical Sciences Improving research reproducibility and reliability 


(UK) 
https://acmedsci.ac.uk/file-download/41615-5836c0640fd92. pdf 
The Welcome Trust Open Research (UK) 

https: //wellcome.ac.uk/what-we-do/our-work/open-research 


Guidelines and recommendations 


NIH Principles and Guidelines for Reporting Preclinical Research 
https://www.nih.gov/research-training/rigor-reproducibili rinciples- 





quidelines-reporting-preclinical-research 

Center for Open Science Transparency and Openness Promotion (TOP) 
guidelines 

https://osf.io/ud578/? ga=1.211230620.829898984.1435325845 
FOSTER Open Reproducible Research 
https://www.fosteropenscience.eu/foster-taxonomy/open-reproducible- 


research 





Proposals: Researchers 


ead eee PERSPECTIVE 
uman e aviour PUBLISHED: 10 JANUARY 2017 | VOLUME: 1| ARTICLE NUMBER: 0021 


OPEN 





A manifesto for reproducible science 


Marcus R. Munafo'*, Brian A. Nosek?*, Dorothy V. M. Bishop®, Katherine S. Button®, 
Christopher D. Chambers’, Nathalie Percie du Sert®, Uri Simonsohn?, Eric-Jan Wagenmakers”, 
Jennifer J. Ware" and John P. A. loannidis'2"3"4 


“Here we argue for the adoption of measures to optimize 
key elements of the scientific process: methods, reporting 
and dissemination, reproducibility, evaluation and 
incentives. There is some evidence from both simulations 
and empirical studies supporting the likely effectiveness of 
these measures, but their broad adoption by researchers, 
institutions, funders and journals will require iterative 
evaluation and improvement”. 


Lakens et al BMC Psychology (2016) 4:24 


aes, UNIVERSITY OF 
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No publication without 
confirmation 


“...a new type of paper for animal studies 
of disease therapies or preventions: one 
that incorporates an independent, 
statistically rigorous confirmation of a 
researcher's central hypothesis. We call 
this large confirmatory study a preclinical 
trial". 


Mogil JS, MacLeod MR. No publication 
without confirmation. Nature. 
2017;542:409-411. 





Drug development: Raise standards for preclinical 


DOI 10.1 186/s40359-016-0126-3 BMC Psychology cancer research 


C. Glenn Begley & Lee M. Ellis 


Affiliations | Corresponding author 


Nature 483, 531-533 (29 March 2012) | doi:10.1038/483531a 


On the reproducibility of meta-analyses: iioi 


six practical recommendations 


Daniël Lakens'”, Joe Hilgard? and Janneke Staaks? 





“...propose how methods, publications and 


incentives must change if patients are to 
benefit”. 





Enhancing reproducibility 


New reporting standards for Nature journal authors are intended to improve transparency andj Kene! £ al. Genome Biology (2015) 16:141 


DO! 10.1 186/s13059-015-07 10-5 Genome Biology 


reproducibility. 


“We will all be using a checklist 
(http://www.nature.com/authors/policies/ 
checklist.pdf) intended to prompt authors 
to disclose technical and statistical 


information in their submissions and to 
encourage referees to consider aspects 
important for research reproducibility”. 
Nature Methods. 2013;10(5):367. 





jove 


Peer reviewed scientific video journal 





httos://www.jove.com 


Proposals: Journals/Editors/Publishers (1) 






© equator 


network 


EDITORIAL Open Access 


Better reporting for better research: a Ou 
checklist for reproducibility 


Amye Kenall'”, Scott Edmunds, Laurie Goodman”, Liz Bal’, Louisa Flintoft®, Daniel R Shanahan’ 
and Tim Shipley* 





New article type verifies experimental reproducibility 


Information Systems journal creates “Invited Reproducibility Paper” 
By Hylke Koers, PhD, and Rebecca Capone Posted on 1: April 2016 


Information Systems, a data 
science journal published by 
Elsevier, has devised a solution 


to the question of 
reproducibility by establishing 
a new article type: the Invited 
Reproducibility Paper. 
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Reporting reviews for publication in the Journal THE ROY AL S O C IETY 
Marie Crowe RN, PhD Professor’ PUBLISHING 





and Charley Baker PhD Lecturer? — Registered Reports 

Version of Record online: 8 MAR 2017 Mental Health Nursing “A Registered Report (RR) is a 

DOI: 10.1111/jpm.12370 ee form of journal article in which 

A methods and proposed 

“Search strategy: some reviews fail to provide sufficient detail to analyses are pre-registered 
enable the replication of the search and others do not provide and peer-reviewed prior to 
eligibility criteria that is linked to the review question/ objectives. The research being conducted 
report of the review must include details of the search strategy, (stage 1). High quality 
the processes used for inclusion and exclusion and a flow diagram with the protocols are then provisionally 
details on identification, screening, eligibility and those studies included”. accepted for publication before 


data collection commences”. 
http://rsos.royalsocietypublishing. 


RESEARCH NOTE org/registered-reports 
CE An Open Science Peer Review Oath 


“Here, we propose steps to help increase the transparency of the scientific method and the reproducibility of 
research results: specifically, we introduce a peer-review oath and accompanying manifesto. These have been 
designed to offer guidelines to enable reviewers (with the minimum friction or bias) to follow and apply 
open science principles, and support the ideas of transparency, reproducibility and ultimately greater 
societal impact. Introducing the oath and manifesto at the stage of peer review will help to check that the 
research being published includes everything that other researchers would need to successfully repeat the work”. 
https://f1000research.com/articles/3-271/v2 





Proposals: Funders 


NHS 


National Institute for 


Journals Library Health Research 


The NIHR is the world’s first health research funder to publish comprehensive 
accounts of its commissioned research within its own publicly and permanently 
available journals. The NIHR Journals Library comprises a suite of five open access 
peer-reviewed journals reporting results from a range of health research areas. 


It aims to increase the impact of NIHR research, so that evidence is shared more 
effectively with the NHS and wider public health landscape. It also improves access 
to the full story of our research projects, to ensure that they can be more easily 
reproduced and that there is a transparent record of the research carried out. To 
achieve this, all project information appears in one location alongside the journal 
report, its summaries and other content produced as part of the project, such as 
articles published in other journals and research data. 


Funding 
wellcome 


Improving the reproducibility of research 


One element of good research practice is reproducibility. To develop 
and implement ways to improve the reproducibility of biomedical 
research, we’ve partnered with the: 


= Academy of Medical Sciences 
= Biotechnology and Biological Sciences Research Council 


= Medical Research Council. 


UNIVERSITY OF 


OXFORD 
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Medical 
Research 


MRC Council 





Reproducibility in science — where the MRC comes in 


by Guest Author on 29 October 2015 


The MRC and a group of partner organisations have today published a report and joint statement about the 
reproducibility and reliability of research, and what can be done to improve them. Here, Jim Smith, MRC Deputy 


Grants & Funding 


NIH's Central Resource for Grant: 


National Institutes of Health 


Office of Extramural Research 





Guidance: Rigor and Reproducibility in Grant Applications 


The NIH is committed to promoting rigorous and transparent research in all 
areas of science supported by a variety of grant programs. Updates to 
application instructions and review language intended to enhance 
reproducibility through rigor and transparency have been implemented for 
research grants and mentored career development awards. Updates 

to institutional training grants, institutional career development awards 
(K12/KL2) and individual fellowships will be forthcoming in 2017 or later. 








Suggested solutions: overview 
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" Study registration 

= Protocol publication 

= Research training: study design and methods 
= Study documentation/recording 

= Reporting guidelines and checklists 

= New article publication types 

= Alternative publication formats 

= Culture change in research institutions 

= Changes to accessibility and research dissemination 
= Changes in incentivisation 

= Awareness raising 


Biomedical librarians and 2o 
research reproducibility @ equator 


network 





= The scientific research process should be highly rigorous to ensure that 
findings are reliable 


= As librarians we aim to support biomedical researchers in eosin 
conducting and publishing research of the highest standard possible 


= In terms of research reproducibility as librarians we can help to 
address the reporting aspects of reproducibility and to a certain 
extent methodological aspects 


= We can use our skills to encourage, support and increase rigor in 
biomedical research 


Research reproducibility in the biomedical sciences is crucial in establishing 
the robustness and reliability of the research results which ultimately 
benefits patients 





So how can librarians help? (1) 
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= Daily contact with biomedical researchers 


= Used to identifying and delivering new services in response to 
changing demands 


= Used to liaising with range of departments and faculty in our 
institutions 


= Work in a diverse range of roles with many opportunities for 
awareness raising and engagement 


Some M actions targeting widely documented bareak and 
reproducibility issues could go a ong way to helping to address 
this serious global problem 





So how can librarians help? (2) 
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WHAT FACTORS CONTRIBUTE TO 
IRREPRODUCIBLE RESEARCH? 


Many top-rated factors relate to intense competition 
and time pressure. 


@ Always/often contribute © Sometimes contribute 


elective reporting 


Pressure to publish 














Librarians can 
help to directly 
address these 
issues 


OW Statistical DOWe 
or poor analysis P 


Not replicated enough 
in original lab 


Insufficient 
oversight/mentoring 


viethods, code unavailabie 
Poor experimental design 


Raw data not available 
from original lab 





Fraud 


Insufficient peer review 
Problems with 
reproduction efforts 


Technical expertise required 
for reproduction: r ! i i 
Variability of i Baker M. 1,500 scientists lift the lid on reproducibility. Nature. 


standard reagents sanay i : 2016;533:452—454. Accessed on 4 May 2017. 
Bad luck : ; : ://www.nature.com/news/1-500-scientists-lift-the-lid-on- 


0 20 40 60 80 100% reproducibility-1.19970 





nature 





So how can librarians help? Examples (3) 


WHAT FACTORS CONTRIBUTE TO 
IRREPRODUCIBLE RESEARCH? 


Many top-rated factors relate to intense competition 
and time pressure. 


è Always/often contribute Sometimes contribute 


elective reporting 


Pressure to publish 


ow statistical powe 
or poor analysis P 
Not replicated enough 
in original lab 







Insufficient 
oversight/mentoring 


viethods, code unavailabie 
Poor experimental design 


Raw data not available 
from original lab 






Fraud 


Insufficient peer review 
Problems with 
reproduction efforts 


Technical expertise required 
for reproduction 


Variability of 
standard reagents 


Bad luck 
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Raise awareness of available reporting 
guidelines 





Sign-post researchers to locally available 
statistical support 





Raise awareness of the importance of 
and initiatives encouraging open science 
and data sharing 





Sign-post researchers to locally available study 
design support 





Baker M. 1,500 scientists lift the lid on reproducibility. Nature. 
2016;533:452—454. Accessed on 4 May 2017. 
http://www.nature.com/news/1-500-scientists-lift-the-lid-on- 





reproducibility-1.19970 






So how can librarians help? Examples (4) 


WHAT FACTORS COULD BOOST 
REPRODUCIBILITY? 


Respondents were positive about most proposed improvements 
but@mphasized trainin@in particular. 


Likely 






@ Very likely 





Better understanding þe 
of statistics [= 










Better mentoring/supervision 


More robust design 
Better teaching 


More within-lab validation 






Incentives for better practice 


Incentives for formal 
reproduction 


More external-lab validation 
More time for mentoring 


Journals enforcing standards 


More time checking 
notebooks 


0 20 40 60 80 100% 
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Highlight key statistical resources and 


online training and advise on local 
statistical support 





Highlight key study design resources and 
online training and advise on local 
research support services 


Work with faculty to encourage/provide 
research design, conduct, reporting and 
statistical training sessions 








Baker M. 1,500 scientists lift the lid on reproducibility. Nature. 
2016;533:452—454. Accessed on 4 May 2017. 
http://www.nature.com/news/1-500-scientists-lift-the-lid-on- 





reproducibility-1.19970 
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So how can librarians help: Methods aan 
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Opportunities for us to help to address methodological aspects of 
reproducibility such as: 


promote the importance of documentation/recording of study 
methods e.g. experimental methods written up in lab notebooks 
raise awareness of available guidance/resources for designing 
research studies 

promote resources/guidance on statistical methods and analysis 
techniques 

encourage researchers to seek local expert help in research 
design and to consult with local statisticians 

advise researchers on available data management, data sharing 
and data storage procedures and guidelines 


Generally raise awareness of documented methodological issues in 
research studies and encourage researchers to think about 
reproducibility at all stages during the research process. 
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So how can librarians help: Reporting Baskin 
@ equator 


network 


= Many more opportunities for us to help to address reporting aspects of 
reproducibility such as: 


— raising awareness amongst researchers about reporting problems 
including selective, unclear, misleading or incomplete reporting and 
highlighting the consequences 

— encouraging researchers to report their study fully including a full 
description of the intervention(s), statistical analysis, study design etc. 

— highlighting the importance of ensuring the results from all research 
studies undertaken are accessible (many studies never publish/report 
their results) 

— encouraging study registration and the development and public 
availability of study protocols 

— raising awareness of and promoting the use of reporting guidelines for 
writing up research studies 





‘Reporting’ is most often associated with formally published reports of research but 
transparent and accurate reporting is just as important when making research 
study reports available on organisational websites, article pre-print services, 
iistitutional repositories etc. and also when sharing research data and methods. 
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= Statements that provide advice on how to report research methods 
and findings 

= Specify a minimum set of items required for a clear and transparent 
account of what was done and what was found in a research study 

= Typically take the form of a checklist, flow diagram or piece of 
explicit text 

= Based on available evidence and reflect the consensus opinion of 
experts in a particular field 

= Complement advice on scientific writing and journals’ instructions to 
authors 

= Some examples include: az CONSORT 


TRANSPARENT REPORTING of TRIALS 






< STROBE Statement 


We f u : 
ihe i Strengthening the reporting of observational studies in epidemiology 


mam PRISMA 


TRANSPARENT REPORTING of SYSTEMATIC REVIEWS and META-ANALYSES 





Example reporting guideline 


checklists 


CONSORT Statement extension for reporting 


wr abstracts of randomized controlled trials 


This extension to the CONSORT Statement provides a minimum list of essential items, that 
authors should consider when reporting the main results of a randomized trial in any journal 
or conference abstract. 


CONSORT for Abstract Checklist www.consort-statement.org 


Identification of the study as randomized 
Contact details for the corresponding author 
Description of the tnal design (e.g. parallel, cluster, non-inferiority) 


Eligibility criteria for participants and the settings where the data 
were collected 
N wocaedacasanls — for gan group 
Clearly defined primary outcome for this report 
S How participants were allocated to interventions 
Whether or not participants, care givers, and those assessing the 
outcomes were blinded to group assignment 


Numbers randomized Number of participants randomized to each group 
Recruitment Traistas CS 
Numbers analysed Number of participants analysed in each group 


effect size and its precision 
[Conclusions General interpretation ofthe results  ć 


Blinding (masking) 
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PRISMA Statement 2009 — Reporting guideline for systematic 
reviews and meta-analyses 


PRISMA stands for Preferred Reporting Items for Systematic reviews and Meta-Analyses. It is an evidence- 
based minimum set of standards for reporting systematic reviews and meta-analyses. It consists of a 27-item 
checklist and a flow diagram which depicts the flow of information through the different phases of a systematic 
review. 

This guideline replaces the existing QUOROM Statement; journals and other organisations are encouraged to 
update their instructions and resources and refer authors to the new PRISMA guidance. 


PRISMA 2009 Checklist www.prisma-statement.org 


Section / topic # Checklist item 


[Tte = |1] Identify the report as a systematic review, meta-analysis, or both. 


ABSTRACT 


Structured 
summary 


CEEE Describe the rationale for the review in the context of what is already known. Oooo 
Objectives Provide an explicit statement of questions being addressed with reference to 


participants, interventions, comparisons, outcomes, and study design (PICOS). 
Protocol and Indicate if a review protocol exists, if and where it can be accessed (e.g., Web 
registration address), and, if available, provide registration information including registration 
number. 
Eligibility criteria Specify study characteristics (e.g., PICOS, length of follow-up) and report 
characteristics (e.g., years considered, language, publication status) used as criteria 
for eligibility, giving rationale. 


Describe all information sources (e.g., databases with dates of coverage, contact with 
sources 


Provide a structured summary including, as applicable: background; objectives; data 
sources; study eligibility criteria, participants, and interventions; study appraisal and 

synthesis methods; results; limitations; conclusions and implications of key findings; 
systematic review registration number. 


study authors to identify additional studies) in the search and date last searched. 
Present full electronic search strategy for at least one database, including any limits 
used, such that it could be repeated. 
Study selection State the process for selecting studies (i.e., screening, eligibility, included in 
systematic review, and, if applicable, included in the meta-analysis). 
Data collection Describe method of data extraction from reports (e.g., piloted forms, independently, in 
process duplicate) and any processes for obtaining and confirming data from investigators. 
List and define all variables for which data were sought (e.g., PICOS, funding 


sources) and any assumptions and simplifications made. 
Risk of bias in 


individual studies 


Describe methods used for assessing risk of bias of individual studies (including 
specification of whether this was done at the study or outcome level), and how this 
information is to be used in any data synthesis. 








EQUATOR Network a 
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Search for reporting guidelines 


thd, Browse for reporting guidelines by selecting one of more of these drop.downs 







Study type Clinical area Section of report 


Or search wath free text 


pee: Laamanen mare Borer aaner 






Reporting guidelines for main 





[s 















































study types j 
Randomised trials CONSORT Extensions Other cuas , 
Observational studies STROBE Extensions Other Displaying 60 reporting gadelines found e om i on 
Systematic reviews PRISMA Extensions Other Key reporting guidelines, shaded green, are displayed frst. Show the most recently :'" a "i 
Case reports CARE Extensions Other itta 
Qualitative research SRQR COREQ Other p a 
Diagnostic / prognostic STARD TRIPOD Other y Ben Pracicen n Oma Analyar and Shanna wo Neuroumaging uang DIR! a ; 
studies tatist ethods and anys 
Quality improvement studies SQUIRE Other somata 
Economic evaluations CHEERS Other Toolkits 
Animal pre-clinical studies ARRIVE eiuci@g This section provides practical help and resources to support you in 
Study protocols SPIRIT PRISMA-P Other 
Clinical practice guidelines AGREE RIGHT Other 





Q Writing research 


See all 359 reporting guidelines 





Q Selecting the appropriate reporting guideline 








Librarian action plan: simple ideas (1) B oeno 
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Librarian Action Plan network 


Librarians have an essential role to play in addressing biomedical research 
reporting and reproducibility issues: a call to action 


Serious concerns have been raised about the robustness of biomedical research 





studies and their resuttant pubtications. Who should I work with? 
Potentis| solutions and recommencitions have Deen much dscusseg, including in the Involve key people from your institution/organisation: Faculty; Deans; Departmental Heads; Research Clinicians. 
Janset Waste Series’ ang in the Maniteste tor Reoroducibie Science” : ae . ae . 
Approach departments: Research Services; Writing Centre; Grants Office, Scholarly Communications Office. 
At the EQUATOR Network’ we Delieve that biomedical librarians have a key rote to . A í 2 ‘ 8 P I 
ptay in haiping to adress thane fundamental cancares. Discuss and share ideas with other biomedical librarians! 


What impect could my library have? 

Siomecics! lidrarians sre excellently pieced to raise awareness of anc target new services towards helping to address 

research reporting anc reprocucibility issues amongst their climiciens/ researchers /fecuty. By taking some simple 

actions now librarians can contribute to reising biomecicel research standards anc pudlicstion quality. The resultant 

higher impact research outputs raise the profile and reputation of institutions, with librarians being seen to neve 

Grectly contributed to this. 

Who should | work with? 

involve key people from your inztitution/orgenisation: Faculty; Deans; Departmental Heads; Research Clinicians. 

Approach departments: Research Services; Writing Centre; Grants Office, Schoterty Communications Office. i 

Discuss and shere ideas with other biomedical librarians! Action plan 


Some simple ideas to get you started! 


Action plan 
Some simple ideas to get you started! PEAN 
Promotion * . estas 
La E AET eee acai aS eee Add/share hyperlinks to initiatives for example the EQUATOR Network, METRICS, COMET, Centre for Open 
Science, NIH Rigor and Reproducibility to brary websites, Linguides or zocisi media. Science, NIH Rigor and Reproducibility to library websites, Libguides or social media. 
aero TEs — ae e “= pane * Raise awareness of reporting guidelines, reproducibility standards, data sharing resources, open access, 
* Highignt reporting and reproducibility issues in brary lesflets and posters. > research study registers in library bulletins and current awareness services. 
Training * Highlight reporting and reproducibility issues in library leaflets and posters. 


* Where relevant adc in a few additional sides about reporting guidelines, research reproducibility, research 

protocols, research study registers, date sharing, open access etc to existing training sessions/workshops. Traini 

* Work with faculty to incorporate training on reporting guidelines and reproducibility into existing courses. raining 

Researcher Support * Where relevant add in a few additional slides about reporting guidelines, research reproducibility, research 
ica Camera acne St aie ee protocols, research study registers, data sharing, open access etc. to existing training sessions/workshops. 
reporting guideline to use up puaicstior. f , ae i Ant Prat EA 

* Disazs current research reporting and reproducibility issues with clinicians/researchers and make sure thet * Work with faculty to incorporate training on reporting guidelines and reproducibility into existing courses. 
they are aware of where they can obtain help in designing, conducting and reporting their research. 





Researcher Support 





1. Reueerch: creasing value, reducing werte, The Lancet Serten. biti /arane chearcet om perien!eapar ch * When providing the results of literature searches to clinicians/researchers advise them on the correct 
2. Masrafé MR et ol A meritestos for reproduce science. Nature humen Behaviour 201 70071. 
us . Contact: Shon Kirtley reporting guideline to use to write up their study for publication. 
3. The EQUATOR Network http / 
EQUATOR Knowledge an 
The EQUATOR Ubrarten Network Mia, coos saute epee om ea 


information Manager * Discuss Current research reporting and reproducibility issues with clinicians/researchers and make sure that 
hoonden they are aware of where they can obtain help in designing, conducting and reporting their research. 








Librarian action plan: simple ideas (2) © OXFORD 


© equator 


= Promotion 

— Link to initiatives e.g. EQUATOR Network, Centre for Open Science, 
Led and Reproducibility from library websites, Libguides or social 
media. 

— Raise awareness of reporting guidelines, reproducibility standards, data 
sharing resources, research study registers in library bulletins or 
current awareness services. 

" Training 

— Add in additional slides about reporting guidelines, research 
reproducibility, research protocols, study registers, data sharing, open 
access etc. to existing training sessions/workshops. 

— Work with faculty to incorporate training on reporting guidelines and 
reproducibility into existing courses. 

= Researcher support 

— Advise clinicians/researchers on the correct reporting guideline to use 
to write up their study for publication. 

— Discuss current research reporting and reproducibility issues with 


clinicians/researchers and make sure they are aware of where to obtain 
help with designing, conducting and reporting their research. 








Librarian targeted action plan (1) 
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Librarians hawe an essential role to play in addressing biomedical research reporting and reproducibility issues 
Can you help? 


Lancet Waste Seriesi Mla to for Reproducible Scenos” 


EQUATOR Hetaork 


Take action to support the Lancet Waste Series and the Manifesto for Reproducible Science 
Ideas for targeted library actions 


= Hiehlieht the importance of protocols, guidelines for writing protocols [2 g. SPIRIT) snd encourage researchers to make ther protocols publicty accessible. 
= 

Wee a TEE lesMets, bulletins, sodal medis 

Trevaing: hoid/incorporste into sasting workshop/training session. Researcher supparit: mention ta researchers) fsqukty during consuttstio»ns. 

= Help neseercteers bo identity ongoing studies investeeting the same ora miar reseerch topic to avoid duplication. 


E Promote research study registers for example for clinical tries jeg. clinicaienals.poy) or systematic reviews [eg PROSPERO], the benefits of mesistenne 
studies, and the requirements for reporting on stucy completion. 

Suggested actions 

Promotion: nighiieht and link to guidelines and initistives on liorary websites, Linguides, lesTets, bulletins, social media. 


mace FESOUITES Supporting research study design, standards for conducting resserdh and research ethics, induding guidance and courses (eg. IRE, MIH]. 
z 


Fromotan handouts include links on library website, bulletins, on sodal medis, Ubguide. Reneercher support: discuss with researchers duning consufietions. 
= Support training in stetistice and research methods 

= 
Promotion: aS Libguide. Training: work with faculty ela et ta R SESSIONS. 





Librarian targeted action plan (2) 


= Promote reporting puideines jeg. CONSORT, PRISMA, STROBE], how bo find then snd how they can be easily used for planning and writing research papers. 
otio 


Suggested otom 

Promotor new kanet poster, Ubguds; inclusion of hyperlinks on library weiste, in bulletins, sodal media. 

Traini: hold reporting,(publicstion workshops; work with faculty to ensure thet reporting guidelines are induded in cows conbent. 

ira rr aera aT alana CRT sri ne oflitersture searches advise mesesrchers on the 


gAs, nduding librarians, in the planning and design of proposed studies (iit search, documentation and reporting). 


Fears cee co ce cg cele chee Pics De tne ened ese ie ec 
involved in the: research team from study impian and the range of support they can provide to researchers and Reseerch Senices Departments. 

= Promote libraniened identification of existing evidence in support of grent so plications and before new reseand is instigsted (ensuring new research 
Suuuested actions 

Researcher suppert: work with faculty and Research Services/Srants Offices to support erent spoliation and the planning and deen of proposed studies. 
= Ereourase reseanchers to obtain the protocols (if available) for incduded studies to check the: protocol comesponds to the content of the published paper. 


Open any a repositories / =e communication 
= Recommend open HES journals to reamhat snd provide advice on open date initiatives, software etc. 
Suggested actions 
Promotor Witeuide: sdd links to website, bulletins, social medis. Troining: hold joint workshop with Research Data Scholar Communications Mansoer. 
= Agvocete the registering, archiving and depasit of nal reseanch reports (and unpublished research meteria) in online institutional repogtones, 

iortions 
Promotor posters; Libeuide: links in bulletins, s208 medi. Researcher support: work with institutional Research Dete/Schoterty Communications Tere. 
= Highlight the issue of metractions as reseeechers are often unaware of retraction notices pasted on publisher websites or im bibliographic databases. 
Suggested actions 
Promotor launch s Current awareness sence specifically hehiehine retracted papers. 


T Promote the node of iniistwes such as PubMed Commons in driving up research standards, induding the reporting and reproducitiliity of iierature searches. 
Sumoested octane. 


References, 
i. Search: incoeming valun, eecucing wake, The Lancet Sarima. bbs ee Lele oe ee be eel 






2 





: Sea a Wature Human Behavicur.2017 021 Contert: Shona Kirtley 2 e © E Q to T 
metwaori 


EQUATOR Knowledge amd Information 
F. Tha EÒUATOR Bletwerk bth Serer poate pets os Man 
The EQUATOR Libeerion Metwork thes! vaw oguna- network on librarian! ger oats SS a aai 
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Librarian targeted action plan (3) 
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Targeted Action Plan takes key recommendations from the Lancet Waste 
Series and the Manifesto for Reproducible Science and provides 
suggested actions that librarians could take to respond directly to each 
set of recommendations. 


Some example actions include: 


Research protocols 
* Highlight the importance of protocols, guidelines for writing protocols (e.g. SPIRIT) and encourage researchers to make their protocols publicly accessible. 


Suggested actions 
Promotion: highlight and link to guidelines and initiatives on library websites, Libguides, leaflets, bulletins, social media. 


Training: hold/incorporate into existing workshop/training session. Researcher support: mention to researchers/faculty during consultations. 


Research study registration 
* Promote research study registers for example for clinical trials (e.g. clinicaltrials.gov) or systematic reviews (e.g. PROSPERO), the benefits of registering 
studies, and the requirements for reporting on study completion. 


Suggested actions 

Promotion: highlight and link to guidelines and initiatives on library websites, Libguides, leaflets, bulletins, social media. 

Training: hold/incorporate into existing workshop/training session. 

Researcher support: mention to researchers/faculty during consultations; insist on study registration before providing help with literature searches. 
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Some example actions (cont.) 


Research design and conduct 
* Highlight resources supporting research study design, standards for conducting research and research ethics, including guidance and courses (e.g. IRB, NIH). 


Suggested actions 
Promotion: handouts; include links on library website, bulletins, on social media, Libguide. Researcher support: discuss with researchers during consultations. 


Training: initiate joint workshop with the local Research Ethics Committee or Research Services Depbrtment. 


Research reporting 

* Promote reporting guidelines (e.g. CONSORT, PRISMA, STROBE), how to find them and how they can be easily used for planning and writing research papers. 
Suggested actions 

Promotion: new leaflet/poster; Libguide; inclusion of hyperlinks on library website, in bulletins, social media. 

Training: hold reporting/publication workshops; work with faculty to ensure that reporting guidelines are included in course content. 

Researcher support: mention to researchers/faculty during consultations; when providing the results of literature searches advise researchers on the 
appropriate reporting guideline to use to write up their study; work with institutional Writing Centres/Scholarly Communications Managers to raise awareness. 


* Promote librarian-led identification of existing evidence in support of grant applications and before new research is instigated (ensuring new research 
addresses uncertainties rather than duplicating previous work). 


Suggested actions 
Researcher support: work with faculty and Research Services/Grants Offices to support grant applications and the planning and design of proposed studies. 
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Role-specific action examples 
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= Research Data Librarian or Open Access Librarian 


* Recommend open access journals to researchers and provide advice on open data initiatives, software etc. 


Suggested actions 
Promotion: Libguide; add links to website, bulletins, social media. Training: hold joint workshop with Research Data/Scholarly Communications Manager. 


Researcher support: highlight to researchers/faculty during consultations. 





= Scholarly Communications Manager/Librarian 


* Highlight the issue of retractions as researchers are often unaware of retraction notices posted on publisher websites or in bibliographic databases. 


Suggested actions 
Promotion: launch a current awareness service specifically highlighting retracted papers. 





= Institutional Repository Manager/Librarian 


* advocate the registering, archiving and deposit of final research reports (and unpublished research materials) in online institutional repositories. 


Suggested actions 


Promotion: posters; Libguide; links in bulletins, social media. Researcher support: work with institutional Research Data/Scholarly Communications Teams. 
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= Discussion: 5-10 minutes 


= Have you got ideas for how you could help address 
research reproducibility in your library? Have you already 
introduced new services/resources in your library? 


= Time to swap ideas and share experiences 


= Short time afterwards for feeding back some ideas 
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From discussions: 
Identified that we have a role to play 
Identified some actions that we could take back to our libraries 


= Ensure that we keep up-to-date with concerns being raised within the 
biomedical research community 


= Proactively identify opportunities where we can expand our services to 
respond to concerns 


= Involve key people from your institution/organisation: Faculty; Deans; 
Departmental Heads; Research Clinicians. 


= Approach departments: Research Services; Writing Centre; Grants 
Office, Scholarly Communications Office. 


= Discuss and share ideas and experiences with other biomedical librarians 





What impact could my library have? 
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= You can demonstrate that your library is: 


introducing new services in the context of responding to important 
international concerns documented in the medical literature 

collaborating widely across the institution including with ae and staff 
in departments such as Research Services, Funding/Grants Offices, 
Academic Writing Centres, Scholarly Communications Offices 

playing a fundamental role in improving the rigour, completeness, 
transparency, quality and reproducibility of the biomedical research 
output of your institution/organisation 

contributing to raising the profile and reputation of your institution as a 
result of improved higher impact research output 


Ultimately you will raise the profile of your library and reassure clinicians, 
researchers and institutional/organisational management that the library, its 
staff and the services provided are proactive, up-to-date and focussed on 


addressing real need. 


As a librarian you will be directly and positively impacting the quality, reliability 


and usability of future biomedical research. 
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Reproducible research is achievable 


Librarians have not been explicitly acknowledged as stakeholders - it is up 
to us to raise our voices and show that we can meaningfully contribute to 
global efforts to address reporting and reproducibility issues 


Take the lead in raising awareness of the issues and promoting available 
solutions 


Encourage scientists to change their practice by providing enhanced 
research support services 


Work with scientists and faculty at our institutions to expand the teaching 
curriculum 


Biomedical librarians do have an important role to play in addressing 
concerns about reporting and a issues and we can make a real 
ifference. 


Expanding library services directly in response to widely documented 
concerns in medical research will raise the profile of the library and will be 
well received by your institutions and organisations. 





Veldkamp LS, Hartgerink CHJ, van Assen MALM, Wicherts JM. Who Believes in the 


Storybook Image of the Scientist?, Accountability in Research. 2017;24:3, 127- 
151, DOI: 10.1080/08989621.2016.1268922 
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“Just like any other professional endeavour involving human beings, 
science is prone to human error and bias...Not only scientists 
themselves, but science policy makers, science funders, academic 
institutes and scientific publishers should all actively strive together 
for a ‘scientific utopia’: a transparent, reproducible science system 
in which there is room for correction of error...It is time to step off 
our pedestal, accept our humanness, and collaborate to create an 
open research culture that acknowledges but at the same time 
addresses our fallibility”. 


+ Biomedical librarians! 


Take action now — we can make a difference! 
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Steering group 


UK EQUATOR Centre 
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Thank You @ cavator 


Action Plans can be freely downloaded from our website. 


Please feel free to contact me with any 
questions or suggestions! 


shona.kirtley@csm.ox.ac.uk 


Librarian Network 





http://www.equator-network.org/librarians 


